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organisms. Artificial Intelligence with deep learning (DL) can now be developed as a 
rapid bacterial classification. The research aims to combine deep learning and support 
vector machines (SVM). The ResNet-101 model of the DL algorithm extracted the im- 
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1. INTRODUCTION 

Bacterial colonies infection is a prevalent disease that infected human bloodstream and made chiefly 
mortality in several countries [I]. In past, microbiologist was classified the bacterial species through conven- 
tional microscope where needs hard skills and takes more time [2], [3]. However, that approach obviously 
could be delayed treatment and antimicrobial therapy that can fatality [4]. Therefore, rapid diagnose is signif- 
icant issues to improve quality of bacterial detection. Current computer-aided diagnosis (CAD) based image 
processing and machine learning proposed bacterial classification in clinical microbiology accurately. Besides, 
CAD utilized several machine learning (ML) approach to classify the various bacterial species that used some 
classifiers such as support vector machine (SVM), backpropagation neural network (BPNN), real AdaBoost, 
and modest AdaBoost [5]-[13]. However, the CAD process includes traditional methods such as handcrafted 
features [14]. Recently, deep learning (DL) a subset of machine learning [15}-[17], that have an advantages for 
medical image analysis|[18]-[26]. The beneficial of deep learning is proposed non-handcrafted approach [17]. 
That utilized numerous convolutional neural network (CNN) layers, it is possible to enhance the classifica- 
tion performance, particularly to improve decision-making. As result, Lopez et al. proposed constructing 3x3 
convolutional layers for mycobacterium tuberculosis image classification which obtained receiver operating 
characteristic (ROC) curve of 99% [27]. In addition, high sensitivity score of 83.78% has achieved to clas- 
sify tuberculosis bacteria images by Kant et al. who developed five-layered fully-convoluted neural network 
architecture combined with a cascading method [28]. Furthermore, Kim et al. has proposed an experiment 
utilizing the CNN model based on the pre-trained DensNet framework, that used to classify bacterial image 
on optical diffraction tomography (ODT) and is achieved an accuracy of 85% [29]. Therefore, CNN architec- 
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tures are proven improving the accuracy of classification performances without handcrafted features approach. 
However, during training the dataset using convolutional layer, that possible generates overfitting and time- 
consuming [30]-[32]. On the other hand, the experimental conducted by Zielinski et al. and Wahid et al. who 
are proposed hybrid DL and ML approach to classify the bacterial species that significant enhance the accuracy 
rate [33], [34]. The advantages of hybrid method is reduce misclassification and rapid processing. In addition, 
the success of the classification task, that can be determined by a robust classifier algorithm. Based on the 
literature, a reliable classifier algorithm is the SVM classifier [8], [34]. Hence, this study proposed to modify 
the hybrid method for enhancing the accuracy classification performance and prediction score. In addition, all 
coding generates on graphics processing unit (GPU) it is to optimize the training process [85], [86]. Further- 
more, this study is also provided comparing performance between the classical transfer learning approach and 
the proposed method. 


2. MATERIAL AND METHODS 
2.1. Dataset preparation 

The datasets were collected from the Parasitology Laboratory at Universiti Sains Malaysia. During 
data collection, the microbiologists provided class labeling of the bacterial species. Figure [I] shows image 
classes of six bacterial organism samples (Burkholderia pseudomallei sp, Hemophilus influenzae sp, Klebsiella 
pneumoniae sp, Pseudomonas aeruginosa sp, Staphylococcus aureus sp, and Streptococcus pneumoniae sp). 
The bacterial samples were provided using the gram stain reaction. The samples are included positive cocci, 
diplococci, and negative bacilli. A Olympus BX40 light microscope series is used to magnify the bacterial 
object on Gram-stained slides under 40x magnification. The datasets are provided 44,985 images with the 
pixel size 244x244 pixels size. 
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Figure 1. Bacterial organism of (a) Burkholderia pseudomallei sp, (b) Hemophilus influenzae sp, (c) 
Klebsiella pneumoniae sp, (d) Pseudomonas aeruginosa sp, (e) Staphylococcus aureus sp, and (f) 
Streptococcus pneumoniae sp 


2.2. Data augmentation 

Recently, DL techniques are most popular in medical fields, such as biomedical imaging [18]. DL 
needs an enormous amount of dataset for the learning process. If the resulting data were too narrow or too 
small, it could add sampling bias towards the bulk of each class [82]. Thus, the DL model with low quantities 
of data can obtain poor performance results [37]. Rachmadi et al. and Mikoajczyk et al. are described the data 
augmentation possible training an efficient model and enhance the learning performance [38], [39]. Therefore, 
this study was conducted data augmentation to enlarge datasets utilizing transformation image techniques such 
as, translation, rotation, scaling, flipping, and reflection. 


2.3. Transfer learning 

Recently, DL perform is provided the transfer learning approach. This is significantly improved com- 
putational for training and validation [40]-[42]. In CAD, the advantages are utilized to reducing data labeling, 
and use existing models [43]. The transfer learning technique is utilized the pre-trained CNN network which 
involved fine-tuning process on convolution layers. That method applied while freeze the first layer and up- 
dating weight on each layer [44]. In the end, the fully-connected and softmax layer is used for classification. 
This study proposed ResNet-101 pre-trained CNN network, that model lead in ILSVRC 2015 classification 
task [45]. 


2.4. Deep feature extraction using CNN 
DL was performed in the deep feature extraction by utilizing a pre-trained CNN model. It works 
by reducing the CNN activation dense dimensionality. Activation layers conduct a deep feature extraction in 
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a pre-trained network as in [44], [46]. This process resulted in feature vectors from the pre-trained weight 
model. The forward-backward iteration during the training process identifies the correct weights. Hence, this 
method can minimize binary cross-entropy (misclassification cost). A pre-trained ResNet-101 model is utilized 
in this study. The proposed method is provided bacterial image classification that it modified the deep feature 
extraction technique. Figure 2 shows the architecture model. The method is extracted features from the early 
convolution layers. It showed on conv! layer that extracted the shallow features. In these case, the feature 
is tend to be shallower, more localized, and have a better spatial resolution on earlier layers. Furthermore, 
the max-pooling layer is added to the lower layer to perform subsampling. Finally, multiple feature maps are 
normalized using local response normalization, later, the new feature was classified. The classification layer on 
pre-trained network was discard and change by using SVM classifier. 


Pre-trained CNN model 
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Figure 2. The proposed method modified CNN deep feature extraction 


2.5. Classifier 
2.5.1. Softmax classifier 

The last step of the generated CNNs model was to extract vectors in fully connected layers. Ad- 
ditionally, the fully connected layers affected the performance of CNN models with a linear feature variable 
was present by [47]. Thus, those features were necessary to be classified using a softmax classifier as a linear 
classifier. On the other hand, a softmax classifier could also be used for multi-class classification in binary 
classification tasks that generalized the logistic regression model. In hinge loss functions describes by [48], a 
mapping function A is denoted as the dot product of x; as data input and weight matrix of w, as (1): 


h(a;,w) = w.xi (1) 


besides, output values were represented as unnormalized log probabilities to class tags, unlike in the hinge loss 
function. Those output values were changed with the cross-entropy loss function as shown in (2) and the output 


in B}. 


ekui 
E; = ee z7; (2) 
k = h(x, w) (3) 


2.5.2. SVM classifier 

An effective classifier is the SVM classifier. Goals for separating of multi-class with the selection of 
a hyperplane significantly increase margin using optimization explained by [27], SVM consists of finding the 
solution: 


N 
= 1 

min F(w,¢) = su w + c> Çi 

w,b,Ci m (4) 
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where N,x,, is a training vector with labels y;. Thus, the weight vector, bias, and slack variable are w,b, 
¢;. Meanwhile, C > 0 is a constant error rate, then y is the non-linear kernel function. At the same time, 
SVM intends to minimize the classification error and maximize the margin size by determining an optimal 
hyperplane. As known, Lagrange multipliers can resolve the optimization issues by modifying the primal 
space to dual space, as (5): 


L(a) = 5 Q; — 3 5 5 XiiYiYjiTi Tj (5) 
n=1 n=1n=1 
which subjects to: 
$ avi = 0,0 > a; > i= 1,2,....0 (6) 
n=1 


in a non-linearity state, y(a;) as a mapping function can replace x into a high dimensional feature. For the 


kernel function, SVM can use k(x,x;) = (xxj) to increase the optimization process. Hence, it can be 
defined as in (7). 
n 1 n n 
L(a) = 5 Qi = 2 5 5 aiaiyiyjk(tixj) (7) 
n=1 n=l n=1 


2.6. Computational hyperparameters 

The domain of hyperparameters is considered to train the dataset using pre-trained CNN models. The 
Adam optimization algorithm as an optimizer is used to optimized for training the models. Instead of the 
classical stochastic gradient descent procedure, the Adam optimizer possibly creates efficient computational 
power and utilizes little memory that showed by [49]. The training of the dataset was conducted by using 
MATLAB software and supported by RTX 2070 Super GPU series with 32 GB memory. Table[I|shows setup 
the hyperparameters setting in proposed models. 


Table 1. Selected hyper-parameters 


Parameter Optimizer Learning rate Epoch  Bacth size 
VGG16 Adam 0.1675 15 32 
VGG19 Adam 0.1875 15 32 
Inception V3 Adam 0.1875 15 32 
ResNet-101 Adam 0.1675 15 32 


2.7. Evaluation performance 

There are numerous efforts in deep learning studies, not all researchers report computational cost, but 
entirely depend on model assessment. Besides, with the confusion matrix explained of [50], that can assess 
each model’s performance. To determine the percentage of each organism that was correctly classified with 
statistical parameters, such as recall (positive sample correctly classified), specificity (negative sample correctly 
classified), and precision (positive class were actually positive). The metric evaluation as follows: 


TP 
recall = TP+EN (8) 
pels TN 
speci ficity = TN 4 FN (9) 
TP 
precision TP de FP ( 0) 


in (8)-{10) explained the number of predictions in which the classifier correctly predicts. TP is defined the 
number of cycles correctly classified as true bacterial, and TN denotes the number of cycles correctly classified 
as false bacterial. FP denotes the number of cycles wrongly correctly classified as false bacterial,and FN 
denotes the number of cycles wrongly correctly classified as true bacterial. 


Rapid bacterial colony classification using deep learning (Son Ali Akbar) 


356 o ISSN: 2502-4752 


3. RESULT AND DISCUSSION 

Recent years, Lamy et al. explained the golden standard used blood culture gram-stained classifica- 
tion based on the shape and color using a microscope [51]. However, those producing are incorrect gram stain 
interpretation from the reader error of 9% to 45% using standardized criteria evaluation [B]. Besides, due to 
the limitation influencing delayed diagnosis and treatment, thus that needs an other approaches to handle the 
problem for expert use. This study develops an AI-based algorithm using deep learning CNN-based for rapid 
bacterial specimen identification. The proposed method is consist to enhance the classification performance and 
reduce time-consuming of diagnosis. In the experiment, the recommended pre-trained models such VGGNet 
(VGG16, VGG19) and InceptionV3 are tested to compare the result between the proposed method 
using ResNet-101 model. On the other hand, the datasets were divided 70% and 30% for data training and data 
testing. The first approach, the transfer learning method is used training and testing the datasets. During train- 
ing, overfitting can be decreased accuracy performance [53]. Hence, the cross-entropy loss function was used 
to minimize loss and optimizing the classification model [48], [50]. The result of the first approach illustrates in 
Figures 3-7, which is shown the training period of accuracy and loss graphs from four pre-trained CNN models 
VGG16, VGG19, InceptionV3, ResNet101. The proposed method in second approach, this study utilized con- 
volution layers to conduct modified deep extraction feature technique, which is the extracted features from the 
early convolution layers will proceed on SVM classifier. Furthermore, Table[2]shows the accuracy achievement 
comparison that obtained from three pre-trained CNN model. The high accuracy performance was achieved the 
proposed method of 99.61% by using ResNet-101 model, which modified deep feature extraction technique. 
In addition, the best processing time in the first approach is 79.20 m/s, but the proposed method is faster than 
the first approach, which is 9.19 m/s (see Table B). 
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Figure 3. The training and validation graph of accuracy and loss per epoch on VGG16 


Figure 4. The training and validation graph of accuracy and loss per epoch on VGG19 model 
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Figure 5. The training and validation graph of accuracy and loss per epoch on InceptionV3 model 
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Figure 6. The training and validation graph of accuracy and loss per epoch on Resnet-101 model 
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Figure 7. The training and validation curve of accuracy and loss per epoch of each model curve line 
description 


Other evaluated performance parameters were recall, precision, and specificity scores. Table/4|shows 
the comparison of both method performances. As shown in Table [4| the first proposed method accomplished 
99.30% recall, 99.37% precision, and 99.88% specificity. Improved performance was shown in the implemen- 
tation result of the second proposed method, which increased up to 28% of recall, 26% of precision, and 9% of 
specificity more than the performance results of the first proposed method. The second proposed method could 
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reduce the number of false-negative detections in clinical diagnosis to reduce excessive anxiety in the patient 
being treated. To performance comparison, the accuracy assessment in image classification was used to evalu- 
ate the proposed training validation set and the CNN model. The proposed models were compared with other 
models which utilized gram-stained images. We evaluated the CNN model with various layers configuration. 
The performance results did not show any significant divergence between models which were previously tested 
in this study. For instance, this stage are tested the dataset using the method proposed [34], which resulted in 
only a slight difference in the accuracy performance, which was 0.74%. Nevertheless, different datasets and 
hyperparameters settings might influence the divergent performance results. Table[5]shows the comparison of 
proposed model with other methods. the proposed methods performed superiority over other models. Hence, 
it could serve as an aided tool for bacterial classification based on Gram-stained images. 


Table 2. The accuracy of classification 
CNN Models Transfer learning (Acc %) Deep feature + SVM (Acc %) 


ResNet-101 99.34 99.61 
Inception V3 98.98 99.44 
VGG19 98.14 98.99 
VGG16 97.78 99.05 


Table 3. Processing time 
CNN Models Transfer learning (m/s) Deep feature + SVM (m/s) 


ResNet-101 118.28 9.19 
InceptionV3 177.00 14.39 
VGG19 147.00 48.20 
VGG16 79.20 47.24 


Table 4. The comparison classification result between CNNs model 


Transfer learning (%) Deep feature + SVM (%) 

2-7 CNN Models Recall Precision Specificity Recall Precision Specificity 
ResNet-101 99.30 99.37 99.88 99.58 99.58 99.97 
InceptionV3 98.90 99.00 99.83 99.42 99.15 99.91 

VGG19 98.22 98.35 99.70 98.87 98.88 99.85 
VGG16 97.62 97.82 99.63 99.02 99.03 99.84 


Table 5. The comparison of accuracies performance between the proposed method and other methods 


Author Methods Accuracy (%) 
H. Sajedi, et al. Gabor Feature, XGboost classifier 90.28 
K.P. Smith, et al. InceptionV3 94.90 
B. Zielinski, et al. VGG, SVM classifier 97.24 
F. Wahid, et al. InceptionV3, SVM classifier 98.70 
Proposed method 1 ResNet-101, softmax classifier 99.34 
Proposed method 2 ResNet-101, SVM Classifier 99.61 


4. CONCLUSION 

In this study, rapid classification based deep learning method has been proposed to improve the predic- 
tion result and fast recognition of bacterial colony species. The datasets are provided of six bacterial species that 
caused patient acute infection symtomps. The proposed method is modified deep feature extraction technique 
using the pre-trained CNN layers, which is extracted features on shallow convolution. Then the classification 
process conducted using SVM classifier. The proposed method utilized the ResNet-101 CNN architecture for 
trained and tested the datasets and compared with three CNN models. As results, the classification perfor- 
mance obtained high accuracy of 99.61% rather then use the transfer learning approach. The prediction score 
involved three matrix parameter measurement that achieved 99.58% recall, 99.58% precision, and 99.97% 
specificity. Besides, the classification time touched fastest time processing of 9.19 m/s. Hence, this study 
has been proven to be successful in enhancing classification accuracy and less time consuming. Furthermore, 
the bacterial colony detector’s success in rapid bacterial species classification set this study apart from other 
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transfer learning studies. In future works, the architecture will be applied for real-time various microorganism 
species detection. 
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